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Met:hod for generating and verifying digital watermarks 
^^^^a nd for exchanging data containing digital watermar ks 

5 Cross References to Related Applications 

This application claims the priority of Euro- 
pean patent application 97810708.4, filed Sept. 26, 1997, 
the disclosure of which is incorporated herein by refer- 
10 ence in its entirety. 

Technical Field 




The present invention relates to methods for 
15 generating and verifying digital watermarks and for 

transmitting data containing digital watermarks according 
to the preamble of the independent claims. 



20 



Background Art 



Digital watermarking is a method for marking 
data sets, such as images, sound or video. A digital wa- 
termark consists of a slight modification of the data set 
that does not affect the data set's usability but that 

25 can be detected using dedicated analysis software or ap- 
paratus. Watermarking can e.g. be used for marking , 
authorship or ownership of a data set. It can also be ap- 
plied for verifying the originality of the multimedia 
data content, where the loss of originality refers to the 

30 degree of contents modification suffered by the image. 

Digital watermarking can be seen as a funda- 
mental-problem in digital communications (see e.g. I. 
Cox, J. Killian, T. Leighton, and T, Shamoon, "Secure 
spread spectrum communication for multimedia" , Proceedings 

3 5 of the IEEE International Conference on Image Processing, 
Lausanne, Switzerland, September 1996) . Early methods of 
encoding watermarks consisted of no more than increment- 
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ing an image component to encode a binary '1' and decre- 
menting to encode a ^0' (G. Caronni "Assuring Ownership 
Rights for Digital Images" in H. H. Brueggemann and W. 
Gerhardt-Haeckl, editors, Reliable IT Systems VIS ^95, 
5 vieweg Publishing Company, Germany, 1995) . Tirkel et al . 
{A. Z. Tirkel, G. A, Rankin, R. G. van Schyndel, W. J. 
Ho, N. R. A. Mee, and C. F. Osborne, "Electronic water- 
mark", in Dicta-93, pages 666-672, Macquarie University, 
Sydney, December 1993) and van Schyndel et al . (A. Z. 
10 Tirkel, R. G. van Schyndel, and C. F. Osborne, ^^a two- 
dimensional digital watermark", in ACCV'95, pages 378- 
383, University of Queensland, Brisbane, December 6-8 
1995) have applied the properties of m-sequences to pro- 
duce oblivious watermarks resistant to filtering, crop- 
15 ping and reasonably robust to cryptographic attack. Ma- 
tsui and Tanaka (K. Matsui and K. Tanaka, "Video- 
Steganography : How to secretly embed a signature in a 
picture", in IMA Intellectual Property Project Proceed- 
ings, pages 187-206, January 1994) have applied linear 
20 predictive coding for watermarking. Their approach to 
hiding a watermark is to make the watermark resemble 
quantization noise. Tirkel and Osborne (see above) were 
the first to note the applicability of spread spectrum 
techniques to digital image watermarking. Since then 
25 there has been an increasing use of spread spectrum in 
digital watermarking. It has several advantageous fea- 
tures, such as cryptographic security (see Tirkel and Os- 
borne, above) , and is capable of achieving error free 
transmission of the watermark near or at the limits given 
30 by the maximum channel capacity (J. Smith and B. Co- 

miskey, "Modulation and information hiding in images", in 
Ross Anderson, editor. Proceedings of the First Interna- 
tional Workshop in Information Hiding, Lecture Notes in 
Computer Science, pages 207-226, Cambridge, UK, May/ June 
35 1996. Springer). Fundconental information theoretic limits 
to reliable communication have been discussed by some 
authors (see Smith and Comiskey, above) . The shorter the 
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payload of a watermark, the better are the chances of it 
being communicated reliably. Spread spectrum is an exam- 
ple of a symmetric key cryptosystem (B. Schneier, "Ap- 
plied Cryptography", Wiley, 2nd edition, 1995). System 
5 security is based on proprietary knowledge of the keys 

(or pseudo random seeds) which are required to embed, ex- 
tract or remove an image watermark. One provision in the 
use of a spread spectrum system is that it is important 
that the watermarking be non-invertible because only in 
10 this way can true ownership of the copyright material be 
resolved (S. Graver, N. Memon, B. Yeo, and M. Yeung, "Can 
invisible marks resolve rightful ownership's 
ISScT/SPIE Electronic Imaging '97 : "Storage and Retrieval 
of Image and Video Databases", 1997). 6 Ruanaidh et al . 
15 (J. K. 6 Ruanaidh, W. J. Dowling, and F. M. Boland, 

"Phase watermarking of images", IEEE International Con- 
ference on Image Processing, Lausanne, Switzerland, Sep- 
tember 1996) and Cox et al . (see above) have developed 
perceptually adaptive transform domain methods for water- 
20 marking. In contrast to previous approaches listed above 
the emphasis was on embedding the watermark in the most 
significant components of an image or a video frame. The 
general approach used in these papers is to divide the 
image into blocks. Each block is mapped into the trans- 
25 form domain using either the Discrete Cosine Transform 
(W. B. Pennebaker and J. L. Mitchell, ''JPEG Still Image 
Compression Standard", Van Nostrand Reinhold, New York, 
1993), the Hadamard Transform (W. G. Chambers, "^Basics of 
Communications and Coding", Oxford Science Publications. 
30 Clarendon Press Oxford, 1985) or the Daubechies Wavelet 
Transform (W.H. Press, S.A. Teukolsky, W.T, Vetterling, 
and B.P. Flannery, ^'Numerical Recipes in C, Cambridge 
University Press, second edition, 1992) . The phase compo- 
nent of the image or video frame is then modified accord- 
3 5 ing to the pseudo-random sequence containing the water- 
marking information . 
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Information can be embedded using the DCT (J. 
J. K- 6 Ruanaidii, W. J. Dowling, and F. M. Boland, Wa- 
termarking digital images for copyright protection", IEEE 
Proceedings on Vision, Image and Signal Processing, 
5 143 (4) : 250-256, August 1996, based on the paper of the 
same title at the IEEE Conference on Image Processing and 
Its Applications, Edinburgh, July 1995) FFT magnitude, 
and phase. Wavelets (see ref s . of Ruanaidh, Bowling and 
Boland, above) , Linear Predictive Coding (see Matsui et 

10 al., above) and fractals (P, Davern and M. Scott, ^^Frac- 
tal based image steganography" , in Ross Anderson, ed. , 
Proceedings of the First International Workshop in Infor- 
mation Hiding, Lecture Notes in Computer Science, pp. 
279-294, Cambridge, UK, May/June 1996. Springer Verlag) . 

15 The key to making watermarks robust has been 

the recognition that in order for a watermark to be ro- 
bust it must be embedded in the perceptually significant 
components of the image (see ref. of Ruanaidh, Dowling 
and Boland, and ref. of I. Cox, J. Killian, T. Leighton, 

20 and T. Shamoon above). Objective criteria for measuring 
the degree to which an image component is significant in 
watermarking have gradually evolved from being based 
purely on energy content (see ref s , of Ruanaidh et al.. 
Cox et al. above), to statistical (see I. Pitas, "A 

25 method for signature casting on digital images". Proceed- 
ings of the IEEE International Conference on Image Proc- 
essing, Lausanne, Switzerland, September 1996) and psy- 
chovisual (see J.F. Delaigle, C. De Vleeschouwer , B. 
Macq, ^'Digital watermarking". Proceedings of the SPIE 

30 Electronic Imaging: Science and Technology, vol. 2659: 
Optical Security and counterfeit Deterrence Techniques, 
San Jose/ February 1996 and M.D. Swanson, B. Zhu and A. 
Tewfik, ^^Transparent robust image watermarking". Proceed- 
ings of the IEEE International Conference on Image Proc- 

35 essing, Lausanne, Switzerland, September 1996) . 

The industrial importance of digital water- 
marking has resulted in a number of products on the mar- 
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ket, either based on spread spectrum techniques or addi- 
tional registration services. They include the Picture- 
marc system by Digimarc (RHOADS, B. Geoffrey, Digimarc 
Corp (US) , "Steganography Systems, WO 96/36163 A, Sure- 
5 Sign (former FBI's Fingerprint) by HighWater Signum (WO 
96/27259) , IP2 system by Intellectual Protocols, the Ar- 
gent system by Digital Information Commodities Exchange, 
the PixelTag system by the MIT Media Lab, the SysCop sys- 
tem from Zhao and Koch by the Frauenhof er-Institut fur 
10 Graphische Datenverarbeitung (J. Zhao and E, Koch, "Em- 
bedding robust labels into images for copyright protec- 
tion" , Proceedings of the International Congress on In- 
tellectual Property Rights For Specialized Information, 
Knowledge and New Technology, August 1995 J, Zhao, "A WWW 
15 Service To Embed And Prove Digital Copyright Watermarks", 
Proc. Of the European Conference on Multimedia Applica- 
tion, Services and Techniques, vol. 2, Louvain-La-Neuve , 
Belgium, May 1996) , and the Tigermark system from NEC 
(European patent Application EP 766468A, Nippon Electric 
20 Corporation (NEC), April 1997) 

The approach proposed by Digimarc (see WO 
96/36163) adds or subtracts small random quantities from 
each pixel according to the least significant bit of each 
pixel compared with the binary mask. The originality of 
25 their approach consists in the use of "subliminal digital 
graticules" that will help in recovering a rotation R and 
a scaling S performed on the marked image. They use an 
exhaustive search strategy based on these graticules to 
recover R and S- This stands in contrast to the template 
30 embodiment described here, where the use of log-polar or 
log- log mapping of the Fourier transform of the image 
combined with cross-correlation in the log-polar or log- 
log plane avoid such a search. 

The Highwater approach (WO 96/27259) describe 
3S a permutation technique to modify the values of the data 
elements according to certain rules which depend on the 
message . 
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The approach of Zhao and Koch, based on the 
JPEG image compression algorithm, proceeds by segmenting 
the image into individual 8x8 blocks. Only eight coef- 
ficients occupying particular positions in the 8x8 
5 block of DCT coefficients can be marked. These comprise 
the low frequency components of the image block but ex- 
clude the mean value coefficient as well as the low fre- 
quencies. Three of the remaining DCT coefficients are se- 
lected using a pseudo random number generator to convey 
10 information. The resemblance of this technique to fre- 
quency hop spread spectrum communications is also men- 
tioned and the blocks are placed at random positions in 
the image. A WWW registration service has been proposed 
for a local registration and a local watermarking, for a 
15 server registration and a server watermarking, and for a 
local watermarking and a server registration. The ap- 
proach is based on a trusted third party model (WWW 
server and Watermark Embedding Gateway) . This model re- 
quires from the Copyright Holder the transfer of relevant 
20 confidential information applied for the watermarking 

process. It is, therefore, possible that the owner of the 
trusted third party system may impersonate the Copyright 
Holder and infringe his copyright. Since the applied key 
for the embedding is not a cryptographic key, copyright 
25 protection and communication security are addressed by 

two different technical solutions, namely the SysCop sys- 
tem and the s-http protocol. These two technical solu- 
tions are applied independently. There is no third party 
verification procedure supported which allows the verifi- 
3 0 cation of the seed, applied for the embedding of the wa- 
termark, by independent parties, such as a court of law. 
The s-http protocol (SSL security protocol) differs from 
the protocol presented below in many aspects (for exam- 
ple, the non- repudiation, security service is not sup- 
3 5 ported by the s-http protocol) . The keys applied for the 
embedding of the mark are furthermore not registered in 
the SysCop system. For copyright verification, the Copy- 
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right Holder has to disclose his key. The information 
generated by the trusted third party is based on the 
cover data, but not on the stego data. 

I. Cox et al from NEC (see EP 766 468, above) 
5 propose to insert watermark into the perceptually sig- 
nificant components of a decomposition of the data in a 
manner so as to be visually imperceptible. In contrast to 
the method described here, they need the original data 
which is compared to the watermarked data to obtain an 
10 extracted watermark. 

J.-F, Delaigle at al , (J.-F. Delaigle, J.-M. 
Boucqueau, J. -J. Quisquater and B. Macq, "Digital Images 
protection techniques in a broadcast framework: An over- 
view". Proceedings of the European Conference on Multime- 
15 dia applications, Services and Techniques, vol. 2, Lou- 
vain-La-Neuve, Belgium, May 1996, J.-F. Delaigle, C. De 
Vleeschouwer & B. Macq, "Digital Watermarking", Proceed- 
ings of the SPIE, vol. 2 659, 1 February 1996) have ap- 
plied signature labeling techniques for the copyright 
20 protection of digital images. The approach presented is 
very similar a EDI security standards. The labeling does 
not influence the multimedia data. Their approach is 
based on an enhanced image format and generates a digital 
signature label in front of the image. This signature la- 
25 bel can be easily overwritten or destroyed. The registra- 
tion entity supports no secure on-line communication pro- 
tocol and is constrained by a legal trusted third party. 
In addition, no means are provided to resolve a conflict 
if multiple watermarks have been embedded in the same im- 
30 age. In an enhanced architecture they propose a general 
watermarking function which uses the output of a hash 
function as the payload of the watermark. This watermark 
function does not support third party verification eind is 
not. based on a spread spectrum technique. In addition, 
3 5 different types of watermarks are not supported. The 

masking scheme presented depend on a ciphering function 
for the inscription. In contrast to the approach pre- 



wo 99/17536 




PCT/IB98/01500 



8 



sented in this disclosure, the secret key has to be re- 
vealed for copyright verification and no coding /decoding 
along with cryptographic digital signatures are applied. 
In addition, the cryptographic key applied is only used 
5 for ciphering and not for other functional purposes rele- 
vant for copyright protection as defined in this disclo- 
sure . 

S. Matyas at all (Stephen M. Matyas , Donald 
B. Johnson, An V. Lee, Rostislaw Prymak, William C. Mar- 
io tin, William S, Rohland, and John D. Wilkins, "EP 0 534 
419 A", Stephen M. Matyas, Donald B. Johnson, An V. Lee, 
Rostislaw Prymak, William C. Martin, William S. Rohland, 
and John D. Wilkins, ^*EP 0 539 726 A") have specified a 
system which is based on an architecture with two dif fer- 
15 ent entities, namely the data processor with a crypto- 
graphic system and the network certification center. The 
overall system security depends on a hierarchical crypto- 
graphic key scheme and digital certificates are only gen- 
erated for a specific data set, called control vectors. 
20 These control vectors set up the basis to identify the 

access rights of users and associated processes they have 
initiated. The main focus of the specified system is the 
enforcement of a dedicated security policy which is based 
on a hierarchical role model. The system is based on a 
25 hardware based security processors and applies symmetric 
and asymmetric cryptographic keys. The cryptographic pro- 
tocols applied are different to the protocols presented 
in this disclosure. The emphasis is to provide a method 
for controlling the use of private and public keys which 
3 0 is not the purpose of our system. In addition, one entity 
needs several different types of keys (symmetric and 
asymmetric) in contrast to our approach which uses for 
one entity one asymmetric key pair only. 

Tanaka et al. (K. Tanaka and K. Matsui, "A 
35 Digital Signature scheme on a Document for MH Facsimile 
Transmission", Electronics & Communications in Japan, 
Part I - Communications, Vol. 74, No. 8, August 1991) 
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propose a digital signature scheme for watermarking fac- 
simile documents (binary images) . This scheme modify the 
length of certain runs of data with a single bit of the 
signature data. 

5 

Disclosure of the Invention 

It is an object of the present invention to 
0 provide a system of the type mentioned above that pro- 
vides a simple and secure way of generating and transmit- 
ting watermarked data. This object is achieved by the 
methods described in the claims. 

In one aspect of the invention, this object 
5 is achieved by an integrated solution method for generat- 
ing and transmitting a data set between two parties H and 
B comprising the steps of a) providing a cover data set 
corresponding to the data set to be transmitted, b) gen- 
erating a stego data set of said cover data set by 
0 embedding at least one digital watermark in said cover 
data set, wherein said watermark is encoded using at 
least one key of an asymmetric cryptographic key pair of 
H, said key pair comprising a secret private key and a 
known public key derived therefrom, and c) encrypting 
5 said stego data set using said key pair of H, d) trans- 
mitting said encrypted stego data set from said party H 
to said party B. 

The party creating the wateirmark can embed a 
detection, a private and a public watermark in the data 
0 set, wherein the detection or the private watermark is 
derived from the private key, the public watermark from 
the public key. The public watermark can be detected by 
third parties while the private watermark can only be de- 
tected using private information. Preferably, the detec- 
5 tion or private watermark is not derived from the private 
key directly but from a hash value of the same and/or 
from a signature generated with the same, such that the 
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author of the watermark does not have to reveal his pri- 
vate key if the private watermark is to be verified. 

In another aspect of the invention, the cover 
data set is provided with a digital watermark and derived 
5 stego data then securely transmitted to a registration 
party that permanently stores at least time information, 
origin of the stego data set, and a digital copyright 
certificate. 

In another aspect of the invention, a tem- 
10 plate modulation pattern is added to the Fourier trans- 
form of an image that is to be provided with a watermark. 
For checking the watermark, the Fourier transform of the 
stego-image is calculated. From this Fourier transform, 
the log-polar mapping transform is generated, which is 
15 then searched for the modulation pattern. Using the log- 
polar transform of the Fourier transform has the advan- 
tage that scaling and rotation of the stego-image are ex- 
pressed in translations- This allows an easy search for 
rotation and scaling using cross-correlation techniques. 
20 However, especially for video data, a change 

of proportion (different horizontal and vertical scaling) 
is more probable than a rotation. In such cases, the tem- 
plate modulation pattern is rather searched in the log- 
log transform of the Fourier transform. Similarly to the 

2 5 log-polar map, the log- log map allows to express the 

horizontal scaling and vertical scaling in translations 
and cross-correlation techniques can be applied to search 
the template. 

In still another aspect of the invention, the 

3 0 image to be watermarked is divided into blocks and the 

magnitude components of the Fourier transform of each 
block is modulated using the same pattern. This method 
provides robustness against cropping of the stego-image 
because a cropping leads* to a circular translation in 
3 5 each block. Preferably, the magnitude components of the 
Fourier transform are modulated, wherein the sign of the 
modulation should be derived from the phase components. 
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thereby reducing interference between the image data and 
the watermark as explained in the following disclosure. 

In a further aspect, the invention consists 
of a method for generating and transmitting a data set 
5 between two parties H and B comprising the steps of pro- 
viding a cover data set corresponding to the data set to 
be transmitted, generating a stego data set of said cover 
data set at a party H by generating at least one digital 
watermark in said cover data set, transmitting a has 

10 value of said stego data set to a registration party, and 
permanently storing certification data at said registra- 
tion party, said certification data comprising said hash 
value of said stego data set, a digital time stamp and 
information designating said party H. 

15 In a further aspect, the invention relates to 

a method for generating a stego data set from a cover 
data set by adding a watermark to said cover data set 
comprising the steps of dividing said stego data sets 
into blocks, calculating a lapped orthogonal transform of 

20 each of said blocks, and applying said watermark to said 
lapped orthogonal transforms. 

In another aspect, the invention relates to a 
method for generating a watermark in a cover data set 
(CD) representing a two or three dimensional data set, 

25 especially for step b) of one of the preceding claims, 

comprising the following steps: A) generating a template 
modulation pattern (T' ) using a random number generator 
seeded by a key (K) , B) calculating the Fourier transform 
of at least part of said cover data set (CD) for generat- 

30 ing Fourier components of said cover data set, C) modu- 
lating at least part of said Fourier components using 
said template modulation pattern (T' ) , D) using the in- 
verse Fourier transform for generating a stego- image 

The invention further relates to a method for 

35 verifying a watermark in a possibly rotated and/or scaled 
version of a two or three dimensional stego data set, 
comprising the steps of: A) calculating a Fourier trans- 
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form of said stego data set (SD) , B) calculating a log- 
polar or a log-log transform of said Fourier transform of 
said stego data set, and C) calculating the correlation 
between said log-polar or log-log transform and a tem- 
5 plate (T) , which template is the log-polar or log-log 
transformation of said watermark. 



Brief Description of the Drawings 

0 

The invention will be better understood and 
objects other than those set forth above will become ap- 
parent when consideration is given to the following de- 
tailed description thereof. Such description makes refer- 
5 ence to the annexed drawings, wherein: 

Fig. 1 the parties involved in individual wa- 
termark protection. 

Fig. 2 the parties involved in watermark pro- 
tection using registered cryptographic keys, 
0 Fig. 3 the parties involved in watermark pro- 

tection using registered cryptographic keys and a regis- 
tration party, 

Fig. 4 the steps taken for embedding a water- 
mark, 

5 Fig. 5 the steps for generating the template. 

Fig. 6 the steps for reading a watermark. 
Fig. 7 the steps for reading the template. 
Fig. 8 the steps for embedding watermark in a 
rotation, scale and translation invariant domain, 
0 Fig. 9. the steps for embedding the watermark 

in an image avoiding to map the original image into the 
rotation, scale and translation invariant domain. 

Fig. 10. the steps to extract the watermark 
from the image, 
i5 Fig. 11 the tiling of the watermark in a 

stego-image or stego video frame, and 
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Fig. 12 the tiling of the watermark in a 
cropped stego-image or cropped stego video frame. 

Modes for Carrying Out the Invention 

5 

I. Terms and Symbols: 

Before describing a preferred method and ap- 
paratus according to the invention, some key te3rms and 
10 symbols used in its description are explained in the fol- 
lowing : 

''Image" : An image in either digital or physi- 
cal form which may constitute a still image or a video 
frame. It can also refer other types of data, such as 
15 video and sound, in particular when being used within the 
context of the protection and owner authentication meth- 
ods of section II of the disclosure. 

"Signal " : A signal in either digital or 
physical form. It may refer to one dimensional or multi- 
20 dimensional signals such as image and video. 

" Copyright Holder (CH) : A party (or a proc- 
ess acting on behalf of it) "owning" a digital image or 
video. This is the party that generates the watermarks, 

'' Buyer (B) " : A party (or a process acting on 
25 behalf it) which obtains (e.g. by purchase) via elec- 
tronic means a specific image from the CH. 

"Stego" : Implies that an image or video data 
is marked. The stego image is also referred to as the 
stego data set (e.g. stego image or video frame) . 
30 "Cover" : Implies that an image or data is un- 

marked. The cover image is also referred to as the cover 
data set (e.g. cover image or video frame). 

"^Watermark" : The form the IAD takes when it 
is in a form suitable for embedding in a signal. 
35 " Copyright Certificate Center (CCC) " : An or- 

ganization (or a process which acts on behalf of it) 
which registers copyright ownership for a specific image 
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or video. Successful registration is only based on a 
sender verification procedure. After successful registra- 
tion a digital copyright certificate can be generated. 
The CCC does not act as trusted third party in our sys- 
5 tem. 

" Digital copyright certificate '' : Digital 
copyright data which comprise the copyright certificate 
data and a digital signature. 

'^ Copyright Request Data (CRD) " : Copyright 

0 data which contains the s t ego- image , the image ID of the 
cover- image, a Universal Copyright Convention Notice, a 
Copyright Symbol, the term Copyright , the year of the 
copyright, the name of the copyright holder, and the 
phrase * 'All Rights Reserved' * . 

5 " Copyright Certificate Data (CCD) " : Copyright 

data which contains relevant copyright information. 

" Digital signature " : A data string which has 
been generated by a cryptographic digital signature gen- 
eration transformation . 

0 " Digital signature generation transforma- 

tion " : A method for producing a digital signature. 

" Digital signature verification transforma- 
tion " : A method for verifying whether a digital signature 
is authentic or not. 

5 " Digital signature scheme " : A scheme based 

on asymmetric cryptographic techniques whose private 
transformation is used for the digital signature genera- 
tion and whose public transformation is used for the 
digital signature verification. 

0 " Digital signature scheme with message recov- 

ery " : A digital signature scheme for which a priori 
knowledge of the input data is not required for the sig- 
nature verification transformation. 

" Digital signature scheme with appendix ^' : A 

5 digital signature scheme for which the input data is re- 
quired as input to the digital signature verification 
transformation . 
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Asymmetric key pair " : A pair of related 
cryptographic keys where the private key defines the pri- 
vate transformation and the public key defines the public 
transformation. 
5 '^ Symmetric key "' : A cryptographic key used 

with a symmetric cryptographic technique and known only 
to a set of specified entities. 

'' Public Key Infrastructure (PKI) " : An organi- 
zation (or processes which acts on behalf of it) which 
10 offers services for the generation, registration, certi- 
fication, distribution, validation, and revocation of a 
certificate associated with an asymmetric key pair. 

" Publ ic watermark " : A watermark that can be 
detected using a publicly available key (or a hash value 
15 thereof) . 

" Private watermark " : A watermark that can 
only be detected using a secret key (or a hash value 
thereof) and some data associated to specific cover data. 
It is not possible for an unauthorized third party to 
20 overwrite or delete the private watermark without the 
cryptographic secret keying information. 

" Detection watermark " : A watermark that can 
only be detected using a secret key (or a hash value 
thereof) . It is not possible for an unauthorized third 
25 party to overwrite or delete the private watermark with- 
out the cryptographic secret keying information. 

"Payload" : The core of the hidden IAD in bit 
foarm without error control coding applied. 

'' Image ID " : The following format scheme for a 
3 0 globally unique ID: The first 3 bytes determine the CCC, 
the following 3 bytes determine the CH ID defined by the 
CCC. Finally the CH can freely assign last 4 bytes for 
each one of his digital images or videos. 

" Oblivious " : A watermarking technique which 
3 5 does not require the cover- image for extracting the mark. 
In other words, only the stego- image is required to ex- 
tract the mark when using an oblivious marking scheme. 
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"' Template '' : A hidden message encoded in the 
image. Two kind of templates are used: "RST template (Ro- 
tation-Scale Template) " and ''PST template" (Proportion- 
Scale Template) . By detecting the RST template, the scal- 
5 ing (zooming) and rotation suffered by a stego-image can 
be deteinnained. By detecting the PST template, the hori- 
zontal and the vertical scaling are detected, and there- 
fore the change of proportion suffered by a stego-image 
can be determined. 
10 " Pseudo random seed " : A value used to ini- 

tialize a pseudo random number generator. 

" Modulation" : Changing a component ' s value 
e.g. by addition or multiplication. 

15 Symbols: 

H, C, B, I 

Distinguished (unique) name of the Copyright 
20 Holder, the Copyright Certificate Center, the Buyer 

B and the Public Key Infrastructure I. 
Cert H , Cert C, Cert B 

Entity H's public key certificate from I, entity 

C's public key certificate from I and entity B's 
25 public key certificate from I. 

(pSx , VSx ) 

The asymmetric signature and verification key pair 
of an entity with the distinguished name X. 
(pcx , VCx ) 

30 The asymmetric decipherment and encipherment key 

pair of an entity with the distinguished name X. 

CC 

A copyright certificate 
DSSMRg (X,Y,Z) 

35 A digital signature generation scheme with message 

recovery, where X denotes the private key, Y the 
input data, and Z the resulting signature . 
DSSMRv(X, Y, Z) 
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A signature verification scheme with message recov- 
ery, where X denotes the public key, Y the input 
data, and Z the resulting output data. 
DSSAPg{X, Y, Z) 

5 A digital signature generation scheme with appen- 

dix, where X denotes the private key, Y the input 
data, and Z the resulting signature. 
DSSAPv(X, Y, Z) 

A signature verification scheme with appendix, 
0 where X denotes the public key, Y the input data, 

and Z the resulting output data, 
crh A collision resistant hash function 
OWEA(X,Y,CD, SD) 

The oblivious, spread spectrum based watermark em- 
5 bedding algorithm with the seed X, the payload Y, 

the cover data CD, and the resulting stego data SD, 
OWVA(X, SD, Y) 

The oblivious, spread spectrum based watermark 
verification algorithm with the seed X, the stego 
0 data SD, and the resulting payload Y) . 

TVP 

Time variant parameter, such as a sequence number 
or a time stamp. 
RPMG(X, Y) 

5 A random phase mask generator, where X denotes the 

cryptographic key as input data and Y denotes the 
resulting phase mask as output data. 
DIES (PM, OI,CD) 

A symmetric digital image encryption scheme, which 

0 is based on the Fourier transform of the image, 

phase modification (random mask encoding by multi- 
plication on the complex exponential component 
^Mm,n) ^ ^ inverse Fourier transform, and quantiza- 
tion, where PM denotes the phase mask and ID de- 

15 notes the original image as input data and OI de- 

notes the ciphered image as output data. 
FFTS(CO, Sr. SMC) 
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A component selector function of the real and 
imaginary FFT components . CO denotes the cover im- 
age, Sr the applied selection rule function, and 
SMC the resulting set of FFT magnitude components. 

5 AF(SMC, HF, MS) 

An authentication function of the selected FFT mag- 
nitude components, where SMC denotes the identified 
magnitude components, HF denotes the applied crh, 
and MS the resulting authentication message as a 

0 string of arbitrary length. For example, AF(SMC, 

HF, MS) consists of generating a string from each 
selected Fourier component, concatenating these 
strings and applying a hash function to the result- 
ing string. 

5 KxY 

A secret key for a symmetric cryptosystem shared 
between two entities with the distinguished name X 
and Y. 
KxY[Data] 

0 denotes the cipher text generated by a symmetric 

cryptosystem with plain text Data. 

1 1 

Concatenation of two data elements , 

CD 

5 Cover Data 

SD 

Stego Data 

0 II. Copyright/Content/Originality protection based on a 
spread spectrum technique 

Depending on the proof-level to be provided 
for the protection, the preferred embodiment of the appa- 
5 ratus and method according to the invention provides 

three different levels of reliability, which are based on 
each other, namely: individual copyright /content /origin- 
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ality protection, copyright /content/originality protec- 
tion with registered cryptographic keys, and copy- 
right/content/originality protection with an CCC on the 
basis of registered cryptographic keys. 

5 Due to commercial requirements, the system 

provides different protection aspects, nameley content 
protection, copyright protection, and originality verifi- 
cation of the stego data. 

The copyright protection of a multimedia data 

0 set is considered as the process of proving the intellec- 
tual property rights to a court of law against unauthor- 
ized reproduction, processing, transformation, or broad- 
casting on the basis of digital evidence data. This proc- 
ess is based on a watermarking process WP and a registra- 

5 tion process RP. RP is executed after WP has been initi- 
ated and finished. RP is executed by a third party, which 
represents a different legal entity as the Copyright 
Holder (CH) , and provides digital evidence data for the 
CH required for verifying copyright ownership. The spe- 

0 cific cover- or stego data is a digital image, or video 
data. The WP embeds or extracts owner authentication data 
in or from multimedia data sets . This owner authentica- 
tion data is embedded such that the commercial usability 
of the multimedia data set is not affected. For this pur- 

5 pose, a key is applied to embed encoded owner authentica- 
tion data, called the watermark, into the cover data set 
I, resulting in a stego data set I*. The watermark data 
can then be extracted from the stego data if the correct 
key is used. 

0 In the following, WP is based on a perceptu- 

ally adaptive spread spectrum technique, a specific type 
of a symmetric cryptographic system. In order to embed or 
extract a watermark, it is necessary to .know the exact 
values of the seed used for the generation of pseudo ran- 

5 dom sequences used to encode the watermark. Because 
spread spectrum signals are statistically independent 
(and therefore virtually orthogonal) , more than one wa- 
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termark may be encoded into the multimedia data set. De- 
pending on the seed applied for the embedding and verifi- 
cation, we distinguish between a private and a detection 
watermark. A private watermark is defined as encoded 
5 owner authentication data embedded with a cryptographic 
signature as the seed. A detection watermark is defined 
as encoded owner authentication data embedded with a 
cryptographic secret key as the seed. We differentiate 
between copyright protection, content protection, and 

10 originality protection. 

Originality protection is considered as a 
process applied after the copyright protection process . 
It enables a third party to check if the image contents 
has been modified on the basis of a public watermark. 

15 Content protection is considered as an addi- 

tional process applied during the trading transaction be- 
tween a service provider and a customer. The content pro- 
tection described is based on the transform domain of the 
image data and not on cryptographic ciphering algorithms 

20 applied during the communication between the service pro- 
vider and the customer, since these cryptographic algo- 
rithms are not robust against loosely compression and 
other image transformations. In addition, the performance 
of ciphering algorithms for the content protection of im- 

25 age or video data is very time consuming. 

The present method and apparatus is based on 
an image or video watermark technique, described below, 
which embeds and detects the the payload of a watermark. 

30 This technique is based on a perceptually adaptive spread 
spectrum technique which provides reliable means of em- 
bedding robust watearmarks . Such a technique will be dis- 
cussed in section III. In addition, a spread spectrum 
techniques is a form of symmetric cryptosystem. In order 

3 5 to embed or extract a watermark, it is necessary to know 
the exact values of the seed used to produce pseudo ran- 
dom sequences used to encode a watermark. The seeds are 
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considered to be cryptographic keys for watermark genera- 
tion and verification. System security can therefore be 
based on proprietary knowledge of the keys and provide in 
addition the necessary security parameters needed for a 

5 secure communication (mutual authentication, integrity, 
confidentiality, non-repudiation) in the trading process 
of digital images or videos. Because spread spectrum sig- 
nals are statistically independent (and therefore virtu- 
ally orthogonal) , the present method and apparatus en- 

0 codes more than one watermark in an image or video frame 
at the same time, namely detection, private watermarks 
and public watermarks. The detection watermark allows to 
identify during a scanning process if the stego data be- 
longs to the copyright material of a CH . The generation 

5 of the private watermark is based on a digital signature 
as the seed and supports, therefore, third party verifi- 
cation who has generated the seed information for the 
coding and the decoding of the payload. The generation of 
the public watermarks enable the verification of the 

0 originality of the received stego data, on the private 
key of the asymmetric key pair of the ICH. 

Since the system provides for the registra- 
tion of the public key of the asymmetric key pair, the CH 
can prove that he is the only person in the possession of 

5 the adequate private key of the asymmetric key pair and, 
therefore, the generator of the private watermarks. 

The system also provides the secure registra- 
tion (mutual authentication, integrity, non-repudiation) 
of watermark encoded images (stego data sets) at a CCC, 

0 The stego-image is registered at the CCC and a digital 

copyright certificate is generated which is signed by the 
CCC. If an unauthorized third party has also encoded wa- 
termarks in the same image, conflicting claims in copy- 
right disputes can be resolved. Examining the time stamps 

5 of the copyright certificate enables the secure identifi- 
cation of the legal owner: The earliest of the time 
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Stamps identifies the legal owner if no copyright revoca- 
tion request has been applied. 

Watermark protection with registered crypto- 
graphic keys and the CCC based copyright protection are 
5 based on a PKI . The PKI issues on request public key cer- 
tificates containing the public key of the party, the 
distinguished name of the party, and a time stamp. Every 
certificate is signed with the PKI ' s private key and the 
trust is built on the validity of the authentic copy of 
10 the PKI • s public key (we assume that the public key of 
the PKI is accessible, authentically distributed, and 
verifiable by every party) , 

In the following three levels of the system 
are described. 

15 

The method described in this section II re- 
quires a suitable watermarking technique. Various such 
techniques are known and can be employed. However, a pre- 
ferred technique is described in the section III. 

20 II. a) Registration based copyright, content, 

and originality protection 

Depending on the proof-level to be provided 
for the protection, our approach provides three different 
protection levels, which are based on each other, namely 

25 individual copyright /content/originality protection, 

copyright/content /originality protection with registered 
cryptographic keys, and copyright /content /originality 
protection with a CCC on the basis of registered crypto- 
graphic keys. Since the first two cases are special cases 

30 of the third one, we present only the approach for the 

registration based copyright protection. Depending on the 
level of protection to be provided (content or original- 
ity or copyright protection) , not all phases described 
below have to be executed. The phases described below 

35 have to be executed for the highest level of protection, 
i.e. content and originality and copyright protection. 
Based on one asymmetric key pair only, H can enforce the 
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different protection mechanisms for copyright, original- 
ity, and content protection. 

As shown in Fig. 3, the system for the CCC 
based protection is partitioned into four processes, 

5 namely the CH with the name H, the B process with the 

name B, the PKI process with the name I, and the CCC pro- 
cess with the name C. Suppose (pSh , vSh ) / (pch /VCh ) , (pss 
,vsb ) . (pCb ,vCb ) , (psi ,vsi ) , (pci ,vci ) , (psc ,vsc ) , and 
(pcc / vcc ) are the asymmetric key pairs of H, B, I and C, 

0 respectively and all the involved parties would like to 
exchange information by on-line communication. (In the 
case of off-line communication, the security mechanisms 
to be provided for the communication are covered by op- 
erational means). H has an authentic copy of Certe and 

5 Certc whose signatures were verified with the authentic 
copy of vsi . B has an authentic copy of Certw and Certc 
whose signatures were verified with the authentic copy of 
vsi . C has an authentic copy of Certw and Certs whose sig- 
natures were verified with the authentic copy of vsi. The 

0 following phases are then applied: 

Phase 1: 

H retrieves the cover data CD, generates a unique identi- 
fier IDcD := crh{H| |SN), where SN is a serial number, 
5 stores IDcD/ and retrieves the key pair (pSh, vSh) . 
Phase 2 : 

Detection watermark embedding (image owner authentication 
and copyright protection) 

H generates the stego data SD applying the transforma- 
0 tion: OWEA(crh(ps„) , SN||SN, CD, SD) . 
Phase 3 : 

Private watermark embedding (copyright protection) 
1. H generates the private Owner Authentication Data 
OADcD applying DSSMR<3(PSh, IDcd/ OADcd) . 
5 2. H generates the stego data SD applying the transfor- 
mation: OWEA(crh(OADcD) / IDcd^ CD, SD) , where CD is 
the SD of the last phase. 
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Phase 4 : 

Public watermark embedding (originality protection) 

1. H generates the set of magnitude components, apply- 
ing FFTS(CD, S, MC) , with the selection function S 

5 and the resulting set MC of the FFT magnitude compo- 

nents. S is given by the normalization of the magni- 
tude components with the JPEG or MPEG quantization 
table entries and constrained by these components 
that will be modified during the coding process of 
10 the public watermark. 

2. H then generates the authentication data for origi- 
nality verification, applying AF(MC, crh, AM), where 
MC denotes the in the last step generated FFT magni- 
tude component set, crh the applied hashing func- 

15 tion, and AM the resulting authentication message as 

output. AM is generated by converting the value of 
every magnitude component into a string and concate- 
nating the resulting strings of every magnitude com- 
ponent into one string. 

20 3. AM is then ciphered with the key pch, i.e. pchIAM] 

and embedded as the payload in the public watermark, 
applying OWEA (crh (vSh) , PCh [ AM] , CD, SD) , where CD is 
the SD of the last phase. 
Phase 5 : 

25 H then stores the resulting stego data SD. 
Phase 6 : 

H and C execute the following steps for the secure regis- 
tration or validation of copyright requests, and the gen- 
eration of copyright certificates. 
30 1. H generates first the copyright request data CRD, 
CRD := crh(SD| |SN) and then the copyright request 
CR, CR := <TD||SigTD>, with TD := CRD | | TVP|||H|tC, 
and DSSAPg(psh, TD, SigTD) . H then transmits CR to C. 
2. C receives CR and verifies TD, applying DSSAPv (vSh/ 
3 5 SigTD, IVR) , where IVR denotes the intermediate 

verification result. If IVR = crh(TD), with TD := 
CRD I |TVP| |h| |C, then TD has been successfully veri- 



wo 99/17536 




PCT/IB98/01560 



25 

fied and the next step shall be executed. In any 
other case, the processing and communication between 
the H and C is stopped. 

3. If verification was successful, C generates the cor- 
5 responding digital copyright certificate executing 

DSSAPg(PSc, CCD, SigCCD) , with CCD:= CRD | j TVP . C then 
stores the copyright certificate CC := CCD] | SigCC 
and generates then the Copyright Confirmation Reply 
CCR, CCR := <TD| lSigTD>, with TD := CC | | TVP ( j C 1 | H, 
0 and DSSAPg(psc, TD, SigTD) . C then transmits CCR to 

H. 

4. H receives CCR and verifies TD, applying DSSAPv(vs„, 
SigTD, IVR) , where IVR denotes the intermediate 
verification result. If IVR = crh(TD), with TD := 

5 CC I I TVP I |C| |h, then TD has been successfully veri- 

fied. H then verifies and stores the CC . The follow- 
ing phase can now be executed repeatedly, if neces- 
sary, without repetition of the previous phases. 
Phase 7 : 

0 H and B execute the following steps for the trading of 
copyright, content, and originality protected digital 
data (images and video) : 

1. B generates the trading transaction Tl , Tl 
<TD| lSigTD>, with TD := IDcd I | TVP | | B | | H , and 

5 DSSAPg(pSb/ TD. SigTD). B then transmits Tl to H. 

2. H receives Tl , verifies TD, applying DSSAPv(vSb, 
SigTD. IVR) where IVR denotes the intermediate veri- 
fication result. If IVR = crh(TD), with D := 

IDcdI I TVP I Ib] |H, then TD has been successfully veri- 
0 fied and the next step shall be executed. In any 

other case, the processing and communication between 
the H and B is stopped, 

3. If the verification was successful, H retrieves with 
the IDcD information the corresponding stego data SD 

5 and generates the trading transaction T2 : = 

<TD||SigTD>, with TD := CD | | TVP | | H | | B, DIES(PM, SD, 
CD) with RPMG(DSSMRg(pSh, B| | SN) , PM) , and DSSAPg(pSh, 



wo 99/17536 




PCT/IB98/01500 



26 

TD, SigTD) . B | | SN designates the B and the picture 
and is called the mask message. H then stores 
DSSMRo(PSh, B||SN) and transmits T2 to B. 
Phase 8 : 

5 B receives T2 and verifies TD, applying DSSAPv(vsh, SigTD, 
IVR) , where IVR denotes the intermediate verification re- 
sult. If IVR = crh(TD), with TD := CD | j TVP | |h| |B, then TD 
has been successfully verified and CD is locally stored. 
Phase 9 : 

10 After B has paid, H retrieves 1Kb and sends vCb[IKb] . B 

receives vcb[IKb]/ deciphers it (pcg [ vcb [ 1Kb] ] ) , and gener- 
ates the random phase mask PM. This random phase mask is 
then used for deciphering CD {DIES(PM, CD, SD) ) to get the 
original stego data SD. 

15 Phase 10: 

B may verify the originality of the stego data SD, re- 
trieving the public key from H and applying 
OWVA (crh (vsh) , SD, pCh [AM] ) . B then deciphers pCh[AM] apply- 
ing vCh [pCh [AM] ] - H then verifies AM applying the same 

20 steps 1 and 2 as described in phase 4. If the verifica- 
tion was successful, the image content has not been al- 
tered. If the watermark has been destroyed or overwrit- 
ten, the contents of the SD has been modified. If the 
verificanion fails, the content has also been modified by 

25 unauthorized parties. 

Remark : 

Depending on the applied asymmetric scheme 
the private decipherment key may be identical to the pri- 

30 vate signature key and the public encipherment key may be 
identical with the public verification key. 

Since the generated asymmetric key pairs are 
unique, the CH can be uniquely identified on the basis of 
the digital copyright certificate. 

35 B may check the copyright certificate re- 

questing C (or H) to transfer an authentic copy of the 
copyright certificate for a given identifier ID^j^. Except 



wo 99/17536 




PCT/IB98/01500 



27 

the data transferred, the applied protocol is the same as 
described above (see phase 6) . 

If H would like to transfer a specific copy- 
right of a CD set to another legal party, he may initiate 
5 a copyright revocation request with C. The different 

phases of this request are analogue to the copyright re- 
quest . 

For copyright verification, the CH first 
verifies the detection watermark and then the private wa- 
0 termark with the extracted SN. 

Copyright verification may be checked by a 
third party, if the H transfers the digital signature ap- 
plied for the seed. Based on the retrived bublic key from 
H, the third party can verify that H is the only one who 
5 has generated the corresponding signature. 

II. b) Copyright, content, and originality 
protection with registered keys 

As shown in Fig . 2 , the apparatus for the 

0 copyright, content, and originality protection with reg- 
istered cryptographic keys is partitioned into three pro- 
cesses, namely the CH with the name H, the Buyer process 
with the name B, and the PKI process with the name I. 
Suppose (psh/VSh) / (PCh/VCh)/ (PSb/ vsb) / (PCb. vcg) , 

5 (psj, vsj) , and (pci,vcx) are asymmetric key pairs of H, 
B, and I, respectively. Suppose H has an authentic and 
actual copy of Certg which signature was verified with 
the authentic copy of vsj and the B has an authentic and 
actual copy of Cert^ which signature was verified with 

0 the authentic copy of vsj. Then the same phases except 
phase 6 as for II. a) have to be applied. 
Remark ; 

Since the generated asymmetric key pairs are 
unique, the CH can be uniquely identified if no addi- 
5 tional watermarks by unauthorized persons have been en- 
coded into the SD . 
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II. c) Individual copyright, content, and 
originality protection 

As shovTn in Fig. 1, the apparatus for the in- 
5 dividual copyright , content, and originality protection 
is partitioned into two processes, namely the CH with the 
distinguished name H and the B process with the distin- 
guished name B. Suppose (psh^vsh) and (pc^. vc^) are 
asymmetric key pairs of H, (psg/VSg) and (peg, vcg) are 
10 the asymmetric key pairs of B. Suppose H has an authentic 
copy of vsg, VC5 and B has an authentic copy of vs^/ vc^- 
Then the same phases as for II. b) have to be applied. 

Remark : 

In the case of a legal copyright dispute, H 
15 can retrieve the payload of the detection watermark and 
construct the signature taken as the seed for the private 
watermark. Since the generation of the same asymmetric 
key pair by two distinguished entities is very unlikely, 
the generation of the digital signature as the seed for 

2 0 the private watermark provides a good level of proof 

against copyright infringement. In the case of watermark 
protection with registered keys, the generation of the 
same asymmetric key pair by two distinguished entities 
can be prevented. 

25 

III. Embedding the watermarks 

The watermarking technique described here 
30 comprises the following steps: 

a) An error-control coding technique for the 
message to be transmitted in the watermark; 

b) A method to encode respectively to decode 
the message resulting from step a) ; 

3 5 c) A reliable method for embedding the en- 

coded message from step b) in the image or video without 
introducing visible artifacts. 
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d) A watermark extraction technique that is 
robust against compression, translation, rotation, scal- 
ing or change of proportion of the stego image or video. 

e) A watermarking technique for small and or 
5 irregular blocks. 

f ) A method that allows to detect if a stego- 
image was marked or not with a given key without extract- 
ing the encoded message. 

g) A method for watermarking without template 
0 which is resistant to translation, rotation and scaling. 

h) A method for watermarking videos . 

Each of these aspects can be applied to con- 
ventional watermarking techniques. Preferably, they are 
used in combination to provide a highly reliable, robust 

5 and powerful method for marking data sets. This method 
can be applied for any watermarking applications, in par- 
ticular to the application described in section II of 
this disclosure. 

Steps a) and b) can be used for embedding wa- 

0 termarks in any type of data while steps c) is optimized 
for embedding watermarks in images or video frames. 

In the following, the above mentioned ele- 
ments of the watermarking technique are described in de- 
5 tail. 

III. a) Error control coding 

Error control coding is applied to the mes- 
0 sage prior to encoding step Ill.b) . When used in combina- 
tion with the procedure described in section II, the mes- 
sage corresponds to one of the blocks BL^ . 

Preferably, symbol based Reed Solomon (RS) 
codes are applied for this purpose. The advantages are 
5 the following: 

- RS codes correct symbol errors rather than 
bit errors, and 
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- RS codes can correct erasures as well as 
errors. Erasures can be factored out of the key equation, 
which means that "'erased" symbols can be ignored. They do 
not play any role in the error control mechanism - an 

5 erasure is useless redundancy. 

Being able to discard erased symbols has two 

advantages : 

- If the posterior probability of a received 
symbol is low, it may be ignored. 

10 - RS codes only come in standard sizes. For 

example a 255 x 8 bit code is common. Most commonly used 
RS error control codes appear to be too large to be used 
in watermarking. However, it is possible to make almost 
any RS code fit a watermarking application by judiciously 

15 selecting symbols as being erased {because they were 
never embedded in the image in the first place) . 

Ill.b) Encoding the message 

20 During encoding, the message to be transmit- 

ted in the watermark is transformed into a form suited 
for being used in the modulation of image components. At 
the same time, it is encrypted using a suitable key. 

If used with the method of section II, the 

25 encoding procedure has access to the cryptographic keys 
Pll and v^ (or their hash values) , which are applied as 
seeds to generate pseudo-random sequences as described 
below. The public key is used for encoding the message of 
the public watermark, the private key is used for the 

30 private watermark. Knowledge of the corresponding key (or 
hash value) is required for recovering the message from 
the watermark. 

A watermark may be embedded or extracted by 
the key owner. In this form spread spectrum is a symmet- 

35 ric key cryptosystem. From the point of view of embedding 
watermarks in images or videos given the cryptographic 
keys the sequences themselves can be generated. A good 
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spread spectrum sequence is one which combines desirable 
statistical properties such as uniformly low cross corre- 
lation with cryptographic security. 

Suppose we are given a message B (e.g. that 
5 was provided with error coding in above step III. a). The 
message has the binary form bib2..-bL/ where b^ are its 
bits. This can be written in the form of a set of symbols 
siS2 • - • - most generally by a change in a number 
base from 2 to B. The next stage is to encode each symbol 

10 Sj_ in the form of a pseudo random vector of length N, 
wherein each element of this vector either takes the 
value 0 or 1. N is e.g. in the order of 1000 to 20000 (in 
the order of 10%-50% of the total number of image coeffi- 
cients (Fourier components) that can, theoretically, be 

15 modulated) . 

In a preferred embodiment, this is carried 
out by using a pseudo random generator seeded by the key 

crh(pH) or crh(VH)- 

To encode the first symbol a pseudo random 

20 sequence v of length N + B - 1 is generated. To encode a 
symbol of values where 0 < s < B the elements Vg, Vg + i 
• • • "^s+N-l extracted as a vector of length N. For 

the next symbol another independent pseudo random se- 
quence is generated and the symbol encoded as a random 

25 vector r2 . Each successive symbol is encoded in the same 
way. Note that even if the same symbol occurs in differ- 
ent positions in the sequence, no collision is possible 
because the random sequences used to encode them are dif- 
ferent - in fact they are statistically independent. Fi- 

30 nally the entire sequence of symbols is encoded as the 
summation: 

m =Si=i. .M ^± 

The pseudo-random vector m has N elements, 
each varying between 0 and M. In a next step, the ele- 
35 ments of m are offset to make their mean zero. These ele- 
ments will determine the strength of modulation of the 
Fourier components of the image in step III.c. 
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When decoding the watermark, a vector m' 
(read-out message) is derived from the stego- image. In 
oblivious watermarking, m' corresponds to the modulated 
Fourier coefficients. Hence, in general m' will not be 
5 equal but "'similar" to m. 

To decode s from m' , the elements of m' are 
first offset to make their mean zero. Then, starting from 
the (known) seed, the first random sequence v of length N 
+ B - 1 is generated and the correlation of v with m' is 
10 calculated. The peak of the correlation indicates the 

offset in the random sequence that was used for gener- 
ating Then, the next random sequence v is generated 
and cross-correlated with m' to retrieve S2 / etc. 

Reliable communications of the apparatus are 
15 best accommodated by using m-sequences or Gold Codes to . 
generate the random sequences and use amplitude modu- 
lation: 

where and r-i_ are bi and in which each bit 0 was 

20 replaced by 1 and each bit 1 by -1 due to the isomorphism 
between the group (exclusive OR, {0,1}) and (*,{1,-1}). In 
this case the values of m are between -M and M. Then the 
decoding is carried out by simply cross correlating with 
each of the random sequences in turn. If the correla- 

25 tion is negative then a binary one has been sent, other- 
wise a binary 0 . 

Gold codes and m-sequences, both insure a 
good reliability and security of the embedded mark. How- 
ever, Gold codes have the advantage that for a given 

30 register length k (N=2*^-l) there is a larger choice for 
the key (2^^-l instead of 2*^-1) and a better correlation 
properties if only part of the sequence is used. 
If M is sufficiently large, the statistical distribution 
of the message m should approach a Gaussian (Central 

35 Limit Theorem) , A Gaussian distributed watermark has the 
advantage that, it is more difficult to detect. The vari- 
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ance increases with order M^/^. in other words, the ex- 
pected peak excursion of the sequence is only order M^/^ 



III.c) Embedding the message in the image or video 

5 

In this step, the encoded message m (e.g. as 
obtained in the previous step) is applied to the image or 
a video for generating the wateirmark. 

In contrast to steps III. a) and Ill.b), em- 

10 bedding the message in the image requires some knowledge 
of the nature of the data stored in the image. In the 
following, the image is assumed to be a two-dimensional 
image that can be a still image or a video frame. The 
method is optimized for robustness against operations 

15 generally applied to images or video frames such as 

translation, cropping, rotating, scaling, change of pro- 
portion. (The method is not optimized for other types of 
data, such as sound or text.) 

In order to achieve robustness against circu- 

20 lar translation, the image block is first subjected to a 
Fourier transform. Then, message m is used to modulate 
the Fourier components. In addition to this, a template 
is embedded in the image, which template can be used for 
detecting rotation, scaling or change of proportion of 

25 the image when reading the watermark. A tiling mechanism 
and suitable phase-dependent correction are applied for 
providing robustness against cropping. 

Figure 4 shows a detailed diagram describing 
the embedding of the watermark. Calculation starts from 

30 the cover image: 

1, If the image is a color image, then compute the lumi- 
nance component (by replacing each pixel by g/2 + r/3 
+ b/5, where g, r and b are its green, red and blue 
components) and use these values for the following 

35 calculations, 

2 . If a predefined block size (Nb)is used, divide the im- 
age into adjacent blocks of size Nb x Nb (e.g. 128 x 
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128 pixels) . otherwise Nb is the minimum of the image 
height and width (Nb =min (height, width) ) . 
Map the image luminance levels (or gray levels for a 
black and white image) because it corresponds to a 
perceptually ''flat" domain by replacing them with 
their logarithm. The logarithm is a good choice be- 
cause it corresponds of the Weber-Fechner law which 
describes the response of the human visual system to 
changes of Iximinance. 

Compute the FFT (Fast Fourier Transform) of each 
block. From the real and imaginary components obtained 
in this way, calculate corresponding magnitude and 
phase components. 

The magnitude components are translation invariant and 
will therefore be used in the following modulation 
steps. (However, it is possible to derive translation 
invariants from the phase spectrum as well, which 
could also be modulated) . 

Select the magnitude components to be modulated. To 
encode a message m of length N, a total n\.imber of N 
components are modulated. In non-oblivious watermark- 
ing, any components can be modulated. For oblivious 
watermarking, because of interference of the cover im- 
age with the watermark, the largest (highest energy) 
components (at about the lowest 10% of the frequen- 
cies) are avoided and components at medixim frequencies 
(about next 30%-50%) are used; these frequencies are 
adjacent and are thus located in a band of frequen- 
cies. These figures are chosen because they generally 
give a good compromise between robustness and visibil- 
ity of the watermark. 

There are several methods for selecting the components 
to be modulated, for example: 

a) The selection of the components to be modulated 
does not depend on the given image. Rather, the 
same components are selected for every image. The 
author as well as the reader of the watermark know 



either the positions of the components to be se- 
lected in advance or a key which allows by means 
of a pseudo-random generator seeded by this key to 
generate the positions. 

b) The largest components (inside the allowable fre- 
quency range) are used for modulation. 

c) Almost all magnitude components in a given fre- 
quency band are used. The upper limit of the band 
is computed such that the number of frequencies 
inside the band be larger than and as close as 
possible to N. 

In the methods b) and c) the order in which the compo- 
nents to be modulated can be provided by a pseudo- 
random generator seeded by a key known by both, author 
and reader . 

When selecting the components to be modulated, care 
must be taken to preserve the symmetry imposed on the 
Fourier components F(k]_, k2 ) by the fact that the im- 
age block is real valued: 

FCki, k2) = F*(Nb " ki, Nb - k2) 
Once the magnitude components (Mi, ... Mjg) to be modu- 
lated are chosen, the corresponding value m-i_ of mes- 
sage m is added to or subtracted from the correspond- 
ing selected magnitude component M-i_ . Addition is used, 
if the corresponding phase component is between 0 
and 7U, subtraction if it is between n and 2n. This 
provides robustness against translation and cropping 
(see below) . 

Before adding /subtracting the values mi to/ from M^, 
the vector m can be scaled to adjust the magnitude of 
its elements to those of the components 

Generally, the elements m^ should be of the same order 
of magnitude as the components . The depth of modu- 
lation or amplitude of the embedded signal should de- 
pend on the objective measure of the perceptual sig- 
nificance. The lower the perceptual significance, the 
higher should be the amplitude of the watermark. 
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Moreover, to insure a good invisibility one can use 
local energy and masking criterion (see J.F. Delaigle, 
C. De Vleeschouwer , B. Macq, "Digital watermarking". 
Proceedings of the SPIE Electronic Imaging: Science 

5 and Technology, vol. 2659: Optical Security and coun- 

terfeit Deterrence Techniques, San Jose, February 
1996) to determine the depth of modulation. However, 
for simplicity, the amplitude for all components is 
kept constant. This constant can be predefined by the 

0 owner or can be some function of the mean and/or the 

variance of the energy in the image or its Fourier 
transform and the values of the pseudo-random vector m 
containing the encoded message (e.g. (mean ( energy ) + a 
* variance (energy) ) /mean (m) , where a is a predefined 

5 constant) . 

6. Add a template by a second modulation of the magnitude 
components. This is described in more detail below. 

7. Compute the inverse FFT using the phase components and 
the modulated magnitude components. 

0 8. Compute the inverse of the perceptual mapping function 
of step 3. For Weber-Fechner law mapping, the inverse 
function is an exponential. 
9 . Replace each watermarked block in the image to obtain 
the snego-image . 

5 10. If the image is a color image, then rescale the red, 
green and blue components by the relative change in 
luminance introduced by embedding a watermark. Typi- 
cally, the red, green and blue pixels occupy a byte 
each in program memory. If overflow or underflow oc- 

0 curs then the pixel is set to the upper bound 255 or 

lower bound 0 respectively. 

Template : 

As mentioned above, a template is added to the image in 
5 step 6. Two kinds of templates can be used: 

a) a RST template - to detect rotations and scaling 
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b) a PST template - to detect horizontal and verti- 
cal scaling. 

The PST template is rather used in case of video frames 
5 (changes of proportion are more likely to occur in the 

case of videos than rotations) and the RST is rather used 
for still images (photographs, paintings, etc, . . . ) . 
The steps for generating the template are illustrated in 
Fig. 5: 

0 20. Apply a log-polar or a log-log map to the magnitude 
components. The log-polar map transforms the magni- 
tude components of the FFT into a polar coordinate 
system (0, log-r) with logarithmic radius axis as 
follows. Consider a point (x,y)GSR^ and define: 

5 x= e^ cos 0 

y= e^ sin 0 

where and 0 < 0 < 271:. If r= e^ \i= log(r) and 

for every point (x,y) there is a unique (0,log(r)) 
that corresponds to it. In the log-polar representa- 

0 tion, a scaling of the image leads to an offset of 

the components along the log-r axis and a rotation of 
the image leads to an offset along the 0 axis. Simi- 
larly, the log- log map transforms the magnitude com- 
ponents into a logarithmic coordinate system (log-x, 

5 log-y) as follows. For each point (x,y)G9t^ define: 




Then, a=log(x) and P=log(y), and in this log-log rep- 
resentation, the horizontal respectively vertical 
0 scaling leads to offsets along the log-x respectively 

log-y axes . 

21. Preferably, low pass filtering is used for interpo- 
lating the frequency space components during this 
mapping. The magnitude components belonging to very 
5 low or high frequencies are not mapped. The following 

modulation is only applied to components in medium 
frequency range. 
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22 . Select the magnitude components in the log-polar or 
log-log coordinate system to be modulated. Typically, 
about 0.1-0.3% of all components are to be modu- 
lated. The RST or PST pattern T formed by the se- 

5 lected components in log-polar or log-log space 

should be such that its auto-correlation under trans- 
lation is weak. For this purpose, the indices of the 
selected components should be coprime or be derived 
from a two-dimensional random sequence. This random 

0 sequence can be generated by a random generator 

seeded by a key K. Whoever knows this key K will be 
able to reconstruct the template and detect the wa- 
termark as explained below. Each selected component 
is increased by a given value. 

5 23. Map the modulated points by change of coordinates 

back into frequency space ( inverse log-polar mapping 
or inverse log -log mapping) . 

The RST or PST pattern T formed by the selected compo- 
0 nents in log-polar respectively log-log space is prede- 
fined and known to the reader of the watermark. 
It must be noted that the calculation of the log-polar 
respectively log-log transform of the cover image or 
video frame is not required for generating the template. 
5 Instead, the RST or PST pattern T of the components to be 
modulated in log-polar respectively log-log space can be 
mapped back to frequency space, which results in a RST or 
PST pattern T' in frequency space that can be applied di- 
rectly to (e.g. added to) the components in frequency 
0 space. Alternatively, the template can be added directly 
in the Fourier transform domain. 

As will be explained below, the template is not required 
for non-oblivious watermarking. 

5 Ill.d) Extracting the watermark from the stego-image or 
video 
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Figure 6 shows a detailed diagram illustrating the steps 
for reading a watermark from the stego- image or stego 
video frame: 

31. If the image is a color image then compute the lumi- 
5 nance component and use these values for the follow- 
ing calculations . 

32. If predefined block size (Nb) is used, divide the 
image into adjacent blocks of size Nb x Nb (e.g. 128 x 
128 pixels) . Otherwise Nb = min (height, width) . 

10 33. Map the image luminance levels (or gray levels) to 
the perceptually ''flat" domain by replacing them 
with their logarithm. 

34. For each block compute the FFT . 

35. Use a data windowing process to suppress the edge 
15 effects in the magnitude spectrum due to possible 

rotation or scaling of the image. Different windows 
can be used such as Blackman, Hamming, Manning, 
Welch or Bartlett Window (see W.H. Press, S.A. Teu- 
kolsky, W.T. Vetterling, and B.P. Flannery, "Numeri- 

20 cal Recipes in C", Cambridge University Press, sec- 

ond edition, 1992) . The effect of data windowing in 
the space domain is equivalent to convolution in the 
frequency domain with a narrow filter. The blurring 
effect introduced by this convolution is beneficial 

25 because it tends to smooth the spectrum which makes 

interpolation more effective. 

36. Determine the rotation and scaling that the image 
suffered by finding the RST template in log-polar 
space or determine the horizontal and vertical scal- 

30 ing by finding the PST template in the log-log 

space. These steps are described below in ''Finding 
the template" section . 

37. Using the results of step 35, read the modulated 
components to generate message m' . This requires the 

35 knowledge of the method that was used in step. 5 for 

selecting, the components to be modulated. 
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Once that the message m' is recovered, it is demodulated 
and error corrected using the methods described in sec- 
tions III. a) and Ill.b). 



5 Finding the template : 

The steps for finding the template are illustrated in 
Fig. 7: 

40. Apply log-polar or log-log mapping to the magnitude 
components of the Fourier transform. The magnitude 

10 components belonging to very low or high frequencies 

are not mapped. The following analysis is only ap- 
plied to components in medium frequency range or to 
all components except the low frequency range. 

41. For oblivious watermarking, calculate the normalized 
15 cross correlation of the components in log-polar or 

log-log space with the RST or PST pattern T that was 
used for generating the template in step 21 and find 
the point of best correlation. If the image has nei- 
ther been rotated or scaled, this point is at zero. 

20 If the image is rotated and/or globally scaled there 

is an offset along the © axis and/or log-r axis, in 
the log-polar map. If the scaling suffered by the 
image or video frame was different on horizontal 
respectively vertical axis, there are offsets along 

25 log-x respectively log-y axes in the log-log map. 

For non-oblivious watermarking, the log-polar re- 
spectively log- log transform of the Fourier compo- 
nents of the cover image can be used instead of RST 
or PST pattern T for retrieving scaling, rotation 

30 respectively change of proportion 

The cross correlation can be calculated efficiently 
using conventional Fourier techniques. 

In order to obtain better results and lower 
computational cost, before applying the cross correlation 

35 one can first adaptively filter the data to remove out- 
liers and noise and use a filter which keeps only local 
peaks. This can e.g. be carried out by locally calculat- 
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ing the variance (or some other value indicative of the 
data's distribution) of neighbouring data of each data 
point. If a given data point lies clearly outside this 
variance, is it replaced by zero. In a next step, local 
5 peaks that have not been filtered out are then stored in 
a sparse matrix to reduce computation. The fast correla- 
tion (using the FFT or by a point by point correlation) 
is done in this case between the peaks of (T) and the 
peaks of (T') . The correlation can moreover be weighted 

10 so that the more reliable central points are more 
strongly weighted. 

It is possible to further increase accuracy 
of the scaling and rotation factors by carrying out the 
following: detecting a scaling and/or rotation in a first 

15 iteration from the correlation between the log-polar or 
log-log transform and the template, using said scaling 
and/or rotation for either a) scaling and/or or rotating 
said Fourier transform, calculating a scaled and/or ro- 
tated log-log or log-polar transform therefrom and corre- 

20 lating said rotated log-log or log-polar transform with 
said template, or b) calculating a second template by 
scaling and/or rotating an original Fourier-space tem- 
plate and calculating a log-log and or log-polar trans- 
form therefrom and using said second template for calcu- 

25 lation a second correlation with said log-log or log- 
polar transform of said stego data 

IlI.e) Embedding watermarks in small and/or irregular 
blocks 

30 

To embed watermark in small blocks, one computes the 
transform over regions that instead of comprising only 
one block, extend over adjacent blocks. To do this one 
can use the Lapped Orthogonal Trans foirm (see H.S. Malvar, 
35 "Signal Processing with Lapped Transf orras " , Norwood, MA, 
1991) which has the advantage to minimize blocking ef- 
fects which would otherwise make a strong watermark based 
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on blocks visible, especially for small block sizes. This 
is followed by the method as described in III.c and 
Ill.d, where the Fourier transformation phase is replaced 
by the Lapped Orthogonal Transform (LOT) application for 
5 the cover image, while keeping the same template opera- 
tions . 

Using small blocks (of roughly 16 by 16 points) allows 
the strength of the embedded message to be modulated as a 
function of the local variance, which renders the method 

10 adaptive. Furthermore the watermark can be recovered lo- 
cally the only requirement being that a sufficient number 
of blocks are available to contain 1 complete message. 
To embed watermark in blocks with irregular shapes 
(non-square and non-rectangular) such as might occur in 

15 MPEG4 video compression, two possible solutions can be 
applied : 

• padding of the irregular blocks in order to obtain 
square blocks, using either constant padding, or sym- 
metrical padding, then method as in III.c and Ill.d; 

20 • avoid the padding phase by directly using wavelet 

transforms of arbitrary length signals (see H.S. Bar- 
nard, Image and Video Coding Using wavelet decomposi- 
tion, CIP-Gegevens , Koninklijke Bibliotheek, Den Haag, 
1994). This is followed by the method as described in 

25 III.c and Ill.d, where the Fourier transformation 

phase is replaced by the Wavelet Transformation for 
the cover image, while keeping the same template op- 
erations . 

30 Ill.f) Watermark detection without extraction 

Being able to detect a watermark without being 
able to decode it is useful and in many cases sufficient 
to 5>rove the identity of the generator of the watermark. 
3 5 This can be done by a Bayesian approach (see J.J.K. 6 Ru- 
anaidh and W.J. Fitzgerald, ''Numerical Bayesian Methods 
Applied to Signal Processing", Series on Statistics and 
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Computing, Springer-Verlag, 1996) that allows to compute 
the probability that a watermark generated by a given key 
is present in the stego-image, relatively to the prob- 
ability that no watermark was generated with that key. 
5 The implementation of this principle operates 

as follows. The used watermark d is a linear combination 
of pseudo-random sequences corrupted by noise: 



d = G b + e 



10 



where e is a noise vector corrupting the watermark, b is 
an M X 1 vector and G is an N x M matrix of bits in form 
+1 and -1 (due to the isomorphism between the group (ex- 
clusive OR, {0,1}) and (*,{1,-1}) 0 was changed to 1 and 1 

15 to -1) . Each column of G is a pseudo-random sequence such 
as an m-sequences or a Gold Code in which 0 was changed 
to 1 and 1 to -1 . 

If we assume that the noise follows a Gaussian 
distribution, the probability that a message of length M 

20 was embedded with a said key k in the stego-image (SD) 
is : 

p(*.MId,S/).-'^"*"r"(M/2)n(N-M)/2)de.(G^G)- 



25 



(N-M)/2 



where F is the gamma function, R5 and are irrelevant 
constants introduced as normalization factors, 

b = (G^Gr'G^d 

and 

f = G^b 



The probability that no message was embedded with the 
said key k in the stego-image (SD) is: 

30 

p{K,\j I a,.>/)oc — — — 

2R„ (d^d)'^'' 
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p(k,0\d,SI) 

and compare with 0 . 

5 lll.g) Watermarking without template 

Using a combination of Fourier transform and a log- 
polar map, i.e. the Fourier-Mellin transform that is the 
Fourier transform of a log-polar map, allows to embed a 

10 watermark in a domain that is invariant to rotation, 

scale and translation, without the need to use a template 
to detect rotations and scaling. The method consists of 
directly transforming the cover- image or -video frame in 
the log-polar domain; the watermark is directly inserted 

15 at this stage. Figure 8. shows the steps for embedding 
the watermark in a rotation, scale and translation in- 
variant domain. 

An alternative which is computationally more efficient 

20 bypasses the mapping of the original image or video frame 
in the rotation, scale and translation invariant domain. 
This is shown in Figure 9 . The scheme to extract the wa- 
termark from the image is shown in Figure 10. 

Replacing the log-polar mapping by the log-log map- 

25 ping allows to embed a watermark in a domain that is in- 
variant to translation, horizontal and vertical scaling. 

This is an idealized watermarking scheme which works 
in principle but which in practice is quite costly and 
difficult to implement. The first difficulty is that both 

30 the log-polar mapping (LPM) and the inverse log-polar 
mapping (ILPM) can cause a loss of image quality. The 
change of coordinate system means that some form of in- 
terpolation must be used. This leads to a second diffi- 
culty, which is rather numerical. Interpolation only per- 

35 forms well if the neighboring samples are of the same 
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scale, which is not verified by the magnitudes of the 
frequency components , 

Ill.h) Watermarking videos 

5 

In the case of uncompressed video each frame 
is marked. One possibility is to use the same key and the 
same watermark in each frame. However this can decrease 
the robustness of the watermark against forgery, There- 

10 fore, it is preferable to use the same key, but a differ- 
ent watermark for each frame (e.g. the label of the video 
followed by the frame number) . In the case of MPEGl or 
MPEG2 compressed video, only the intraframes I (the first 
frame of each group of pictures) are marked. 

15 Another novel alternative for watermarking 

uncompressed video is to individually mark three- 
dimensional spatio temporal blocks of video stream, which 
may be overlapped in time and/or in space. The method 
used here is an extension of the algorithms used for 2D 

20 images to the temporal dimension, using 3D Fourier 

transform, 3D template, and the same spread spectrum 
techniques to generate the watermark. The use of Fourier 
transform ensures the same rotation, scaling, and pro- 
portion invariances . We have also a full invariant 3D wa- 

25 termark for theses blocks, exactly as for 2D still image 
watermarking. These 3D blocks may be rather large, or 
small enough to ensure more robustness against cropping. 
As for individual frame marking, we can use the same wa- 
termark for all blocks, or a different watermark for 

3 0 each block. The advantage of this spatio temporal ap- 
proach is to take in account the motion and scene varia- 
tion in watermarking, as developped in the paper of M.D. 
Swanson, B. Zhu and A.H. Tewfik, "Multiresolution Scene- 
Based Video Watermarking using Perceptual Models", IEEE 

3 5 Journal on Selected Areas in Communications, vol. 16, no. 
4, May 1998. However, in contrast with our apparatus. 
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they make use of ID temporal wavelets transform instead 
of our 3D Fourier transform. 



5 IV. Properties of the watermark: 

In the following, some of the properties of 
the watermark generated using the steps described above 
are discussed. 

10 

Resistance to cropping: 

One feature of translation invariants devel- 
oped using the Fourier transform is that they are invari- 
ant to circular translations (or cyclic shifts). This is 
15 used to construct watermarks that are invariant to crop- 
ping. This is illustrated by reference to Figs. 11 and 
12. 

As mentioned above, the image is split into 
blocks and the watermark is applied to each block. In 
20 other words, the same modulation pattern is applied to 

the Fourier components of each block, wherein the modula- 
tion pattern is given by the corresponding encoded mes- 
sages m. 

Fig. 11 shows such an image where the fat 
25 lines 100 designate the borders between the blocks. Sup- 
pose that the watermark in a standard size block will be 
of the form: 

T=[A B ; C D] 

where the sub-matrices A, B, C and D are of 
30 arbitrary size. A circular translation of such a water- 
mark is of the form: 

S= [D C ; B A] . 

The original stego- image is tiled^ with water- 
marks in the pattern [TTTT ; TTTT ;TTTT]. There- 
35 fore, a cropped section of the matrix will carry a water- 
mark in the form [SSSS ; SSSS ; SSSS]. This is 
illustrated in Figure 12. When reading the watermark of 
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the cropped image of Fig. 12, each block carries the wa- 
termark S. Since S is a circular transform of it can 
be read without problems in the Fourier domain using the 
steps outlined above. 
5 Note, however, that the cover image is not 

tiled, only the watermark is. Therefore, while cropping 
merely induces a circular translation of the watermark in 
each block, the change of image in each block is not a 
circular translation. To compensate for this, the phase 

10 components of the Fourier transform must be used for 
correcting the sign of the modulation of the magnitude 
components M-^ , as it is outlined under step 5 above. 

The optimum size of block depends on a number 
of different factors. A size that is a power of two is 

15 useful because the FFT can be used. The block size also 
must be small enough to withstand cropping but large 
enough to comfortably contain a watermark. The best com- 
promise for block size is 128. 

20 Resistance to scaling and rotation: 

As mentioned above, reading the RST template 
in log-polar space allows to detect and measure any scal- 
ing and/or rotation that was applied to the image. This 
information can then be used for reading the watermark. 

25 Since the reader knows the pattern that was used for 
modulating the magnitude components in step 5, he can 
identify the modulated components in the scaled and ro- 
tated image and derive the message m' therefrom. An al- 
ternative is to compensate the transformation using the 

30 measured rotation and scaling and read the message in the 
compensated image . 

Note that the apparatus does not explicitly 
use a rotation and scale invariant watermark but instead 
searches the parameter space of rotations and scales. 

35 Since searching the space of rotation and scales in the 
frequency or space domain is quite complicated (as e.g. 
described in the WO 96/36163), the log-polar map is used 
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where these parameters are Cartesian coordinates and can 
be searched using efficient correlation techniques. 
Resistance to change in aspect ratio: 

Similarly as above, reading the PST 'template 
in log-log space allows to detect and measure the hori- 
zontal and vertical scaling that was applied to the im- 
age or video frame. This information can then be used to 
compensate the transformation, which the allows the wa- 
termark to be read. 



The use of the log-polar map (LPM) or log-log 
map (LLM) changes depending on whether the watermark was 
inserted block by block of predefined size in the FFT do- 
main or whether the block size depends on the image size. 

15 In the first case, the LPM or LLM is used to detect scale 
changes in the image. In the latter case, the maps are 
used to detect the ratio between the FFT size used in em- 
bedding (which is unknown since the original image size 
is unknown in oblivious watermarking) and the FFT size 

20 used in extraction, which equals the size of the image in 
which we attempt to extract the watermark. This is impor- 
tant in cases where the image size has changed as a re- 
sult of e.g. cropping or rotation since the relative po- 
sitions of the FFT points change. 

25 

Lossy compression: 

The robustness of the watermark to operations 
such as lossy compression is achieved by using a percep- 
tually adaptive spread spectrum communications approach, 
30 in which a spread spectrum signal is embedded in selected 
components of the magnitude spectrum of the Fourier 
Transform of the image. 

Redundancy : 

3 5 The watermark is embedded in blocks of a 

fixed size with exactly the same watermark embedded in 
each block. This means that the watermark can be recov- 
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ered from a single block only. This leads to a redundancy 
that increases the chance of extracting the watermark 
correctly from more than one block. 

5 V . Siimmary 

The following summarizes some of the proper- 
ties of the preferred embodiments of the invention. 

The use of an asymmetric cryptographic key 

0 pair for the seed generation enables the execution of 

asymmetric key agreement protocols with message recovery 
or appendix and the protection of the communication be- 
tween the involved parties. Different security services 
for the communication, such as mutual authentication, in- 

5 tegrity, confidentiality and non-repudiation are sup- 
ported by the system with one asymmetric cryptographic 
key pair of the watermark author only for a registration 
or trading process 

The present technique enables a strong bind- 

0 ing relation between the image ID, the image, and the CH 
if the CH registers his copyright at the CCC. If an image 
is watermarked later by an unauthorized person, the time 
stamp in the copyright certificates resolves the copy- 
right ownership. 

5 The CH does not have to reveal his private 

cryptographic key if ownership verification has to be ap- 
plied by a different legal party. 

The present technique supports transferal of 
copyrights. If copyright is transferred to another legal 

0 party, corresponding copyright revocation certificates 
may be generated . 

Digital signatures techniques are applied for 
the security of the communication between different par- 
ties, and the authentication data embedded in a private or 

5 public watermark of an image or video. No signature la- 
beling techniques of the complete image or video are ap- 
plied by the system. 
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In addition, originality protection and image 
content protection by ciphering/deciphering in the trans- 
form domain is supported. 

The Fourier Mellin transform is the Fourier 
5 Transform of a log-polar map. It allows to embed a wa- 
termark in a domain that is invariant to rotation, scale 
and translation. However this approach is costly and dif- 
ficult to implement, and therefore it has been enhanced 
by combining with a Fourier Transform based template em- 
10 bedding technique . 

In the present invention, the log-polar map 
of a Fourier transform is used as a means of facilitating 
rotation and scaling invariance. In order to be invariant 
to scaling and change of proportion, the log-log map of 
15 the Fourier transform is also used. 

Circular translation invariants are used as a 
means of constructing digital watermarks that are invari- 
ant to cropping. 

In contrast to some known techniques, the 
20 present system does not require a database of all water- 
marks that were ever embedded in image anywhere . 

Information is embedded and/or retrieved in 
the log-polar or log-log domain of the Fourier transform. 
Frequency components are modulated which are oblivious to 
25 the cover image but which also have the property that 
they form an unambiguous non-repeated pattern in log- 
polar respectively log-log space. They are used for de- 
termining the degree of rotation and scaling respectively 
the change of proportion suffered by a stego-image in the 
30 absence of the cover-image. Coprime frequencies are use- 
ful for generating such a pattern or template. Uniform 
random sampling of log-polar or log-log space is another 
method that can be applied. 

The technique applies a new concept of in- 
35 variants which eliminate the need for explicitly search- 
ing for rotation and/or scaling values. 
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The methods described above can be incorpo- 
rated into an apparatus, such as one or more computers, 
using know programming and hardware techniques. To prove 
the feasibility of the approach, a Java based copyright 
5 protection and authentication environment for digital im- 
ages has been implemented. The PKI , the CH, the CCC, and 
the IB application processes all implement a Graphical 
User Interface and a server, supporting both console us- 
ers and other requests through a socket interface. 
-10 While there are shown and described presently 

preferred embodiments of the invention, it is to be dis- 
tinctly understood that the invention is not limited 
thereto but may be otherwise variously embodied and prac- 
ticed within the scope of the following claims. 



15 



